WWC Intervention Report 




U.S. DEPARTMENT OF EDUCATION 


What Works Clearinghouse "ies&w.^ 


Dropout Prevention 




April 16, 2007 




High School Redirection 


Program description 


High School Redirection is an alternative high schooi program 
for youth considered at risk of dropping out. The program 
emphasizes basic skills development (with a particuiar focus 
on reading skills) and offers limited extra-curricular activities. 
The schools operate in economically disadvantaged areas and 


serve students who have dropped out in the past, who are teen 
parents, who have poor test scores, or who are over-age for 
their grade. To foster a sense of community, the schools are 
small and teachers are encouraged to act as mentors as well as 
instructors. 


Research 


Two studies of High School Redirection met the What Works 
Clearinghouse (WWC) evidence standards and one study met 
WWC evidence standards with reservations. The three random- 
ized controlled trials included more than 1,600 students in 
Stockton, California; Wichita, Kansas; and Cincinnati, Ohio.^ 


The WWC considers the extent of evidence for High School 
Redirection to be moderate to large for staying in school, for 
progressing in school, and for completing school. 


Effectiveness 


High School Redirection was found to have mixed effects on staying in school, potentially positive effects on progressing in school, 
and no discernible effects on completing school. 




Staying in schooi 


Progressing in schooi Compieting schooi 




Rating of effectiveness Mixed effects 

Improvement index^ Average: +6 percentile points 

Range: -5 to +20 percentile 
points 


Potentially positive effects No discernible effects 

Average: +4 percentile points Average: +4 percentile points 

Range: -3 to +10 percentile Range: -1 to +8 percentile 

points points 



1. The evidence presented in this report is based on availabie research. Findings and conciusions may change as new research becomes avaiiable. 

2. These numbers show the average and range of improvement indices for aii findings across the studies. 
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The High School Redirection study that this intervention report 
summarizes was prepared by staff of Mathematica Poiicy 
Research, Inc. (MPR). The principal investigator for the WWC 
dropout prevention review is an MPR staff member and was 
the iead author of the High School Redirection study. For this 



reason, the study was rated by staff members from Caliber, an 
ICF International Company, who also prepared the intervention 
report. The report was then reviewed by MPR staff members, 
as well as members of the WWC Technical Review Team and 
external peer reviewers. 



Developer and contact 

High School Redirection is no longer an active program and no 
current developer or contact information is available. Additional 
information about the program model and the implementation 
experience of districts that have used it can be found in the 
Baker and Weinbaum (1991) and Rubenstein (1995) reports listed 
in the “References” section of this report. 

Scope of use 

The original High School Redirection opened in 1968 as an alter- 
native high school and was operated by the New York City public 
school system until it closed in 2004. In 1987, the U.S. Depart- 
ment of Labor launched the Alternative Schools Demonstration 
Program (ASDP), which replicated the High School Redirection 
model in seven urban school districts in six states. Because of 
budget pressures, shifting district priorities, and other factors, 
these ASDP evaluation schools have all since closed or are no 
longer following the High School Redirection model. Although 
these programs have ended. High School Redirection’s core 
features— small size, access to child care, and emphasis on 
academic remediation and accelerated credit accumulation— are 
shared with many alternative high school programs currently 
operating throughout the country. 

Description of intervention 

High School Redirection is an alternative high school model 
for youth who are at high risk of not completing school. The 
program emphasizes basic skill development and offers an 
intensive remedial reading program. Strategies and Techniques 



for Advancement in Reading (STAR), for students with serious 
literacy problems. In many cases, the schools offer opportunities 
for independent study and accelerated credit accumulation, so 
that students who have fallen behind can make quick progress 
toward graduation. Beyond the emphasis on basic reading 
skills, the program model does not require specific curriculum 
elements; the schools generally follow the standard curriculum 
offered by their school district. To help create a sense of com- 
munity, schools are relatively small— typically with no more than 
500 students. Teachers are encouraged to act as mentors as 
well as instructors and classes are kept small to allow for more 
individualized attention. The program model calls for on-site 
child care and limited extracurricular activities. The schools 
issue standard high school diplomas, operate in low-income 
areas separate from other high schools, and maintain a degree 
of autonomy from the local school district in day-to-day policy- 
making and staff selection. The schools target former dropouts, 
teen parents, those who are over-age for their grade, and 
students with low test scores. 

Cost 

The cost of High School Redirection per student per year of 
program participation was estimated to be $3,455 in Cincinnati 
and $1,067 in Wichita.® These estimates represent the additional 
cost of the program beyond the cost per year of educating 
the student in a regular school within the district, which was 
estimated to be $7,256 and $5,686, respectively. No information 
is available for Stockton. 



3. See Weinbaum, A. T., & Baker, A. M. (1991). Costs have been converted to 2006 dollars using the Consumer Price Index. 
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Research 



Effectiveness 



WWC Intervention Report 



Five studies reviewed by the WWC investigated the effectiveness 
of High School Redirection. Three studies were included within 
one research report (Dynarski & Wood, 1997). This report sum- 
marized findings on the effectiveness of the replications of High 
School Redirection sponsored by the U.S. Department of Labor 
as part of the ASDP evaluation. Two of the Dynarski and Wood 
(1997) studies— those conducted in Wichita, Kansas, and Cincin- 
nati, Ohio— met WWC evidence standards. A third— conducted 
in Stockton, California— met evidence standards with reserva- 
tions. The Stockton study received a lower rating because a 
substantial number of control group students enrolled in the 
intervention school."* The remaining two studies of High School 
Redirection did not meet WWC evidence screens. 

The Dynarski and Wood (1997) studies in Stockton, Wichita, 
and Cincinnati were all randomized controlled trials in which appli- 
cants to the alternative school were assigned either to the inter- 
vention group, who were offered admission to the school, or to 



a control group, who were not. The Stockton study included 374 
youth who applied to enter the program for the 1991-92 school 
year; the Wichita study included 358 applicants for the 1991-92 
and 1992-93 school years; and the Cincinnati study included 902 
applicants for the 1993-94 and 1994-95 school years. 

Extent of evidence 

The WWC categorizes the extent of evidence in each domain as 
small or moderate to large (see the What Works Clearinghouse 
Extent of Evidence Categorization Scheme) . The extent of 
evidence takes into account the number of studies and the 
total sample size across the studies that met WWC evidence 
standards with or without reservations.® 

The WWC considers the extent of evidence for High School 
Redirection to be moderate to large for staying in school, for 
progressing in school, and for completing school. 



Findings 

The WWC review of interventions for dropout prevention 
addresses student outcomes in three domains: staying in 
school, progressing in school, and completing school. 

Staying in school. One study of High School Redirection showed 
statistically significant effects on staying in school. The Stockton 
study indicated that High School Redirection youth were enrolled 
39 more days on average in the first follow-up year than control 
group youth (110 days versus 71 days) and 17 more days on 
average in the second follow-up year (67 days versus 50 days). In 
addition, at the end of the third follow-up year, fewer High School 
Redirection youth had dropped out (43% versus 53%).® The other 
two studies showed no statistically significant or substantively 
important effects of High School Redirection on staying in school. 



Progressing in school. The Stockton study found that, at the 
end of the fourth follow-up year. High School Redirection youth 
had, on average, earned more credits toward graduation than 
control group youth— 10.5 versus 8.5 credits— a difference that 
was statistically significant. The Wichita study found no statistically 
significant effect on total credits earned. The Cincinnati study did 
not examine outcomes associated with progressing in school. 

Completing school. The Stockton study found that, by the end 
of the third follow-up year. High School Redirection youth were 
more likely than control group youth to have completed high 
school or earned a GED certificate— 40% versus 32%— a differ- 
ence that was not statistically significant. The Wichita and Cincin- 
nati studies found no difference between the research groups in 
completing school at the end of the second follow-up year.'' 



4. Since some Stockton control group students attended the intervention school, the estimated effects of High School Redirection from the Stockton study 
are somewhat understated. 

5. The Extent of Evidence Categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the 
number and size of studies. Additional factors associated with a related concept, external validity, such as students’ demographics and the types of 
settings in which studies took place, are not taken into account for the categorization. 

6. This difference was statistically significant at the 0.10 level but not at the 0.05 level, the standard used for statistical significance by the WWC. 

7. The Cincinnati study did find that High School Redirection had a small, statistically significant positive effect on receiving a high school diploma; how- 
ever, the intervention did not have a significant effect on the combined outcome of receiving either a high school diploma or a GED certificate. 
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Effectiveness (continued) 



The WWC found High School 
Redirection to have mixed 
effects on staying in schooi, 
potentiaiiy positive effects 
on progressing in schooi, 
and no discernibie effects 
on compieting schooi 



References 



Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as: positive, potentiaiiy positive, mixed, no discernible 
effects, potentially negative, or negative. The rating of effective- 
ness takes into account four factors: the quality of the research 



design, the statisticai significance of the findings,® the size of 
the difference between participants in the intervention and the 
comparison conditions, and the consistency in findings across 
studies (see the WWC Intervention Rating Scheme) . 



Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and 
an average improvement index across studies (see Technicai 
Detaiis of WWC-Conducted Computations) . The improvement 
index represents the difference between the percentile rank 
of the average student in the intervention condition versus 
the percentile rank of the average student in the comparison 
condition. Unlike the rating of effectiveness, the improvement 
index is based entirely on the size of the effect, regardiess of 
the statistical significance of the effect, the study design, or the 
analyses. The improvement index can take on values between 
-50 and +50, with positive numbers denoting results favorable to 
the intervention group. 

The average improvement index for staying in school is 
+6 percentile points based on three studies, with a range of 



-5 to +20 percentile points across the findings. The average 
improvement index for progressing in school is +4 percentile 
points based on two studies, with a range of -3 to +10 percentile 
points across the findings. The average improvement index for 
completing school is +4 percentile points based on three stud- 
ies, with a range of -1 to +8 percentile points across the findings. 

Summary 

The WWC reviewed five studies of the effectiveness of High 
School Redirection. Two studies met WWC standards, a third 
met evidence standards with reservations, and the other two 
did not meet WWC evidence screens. Based on the results from 
the three qualifying studies, the WWC found mixed effects on 
staying in school, potentially positive effects on progressing 
in school, and no discernible effects on completing school. 

The conclusions presented in this report may change as new 
research emerges. 



Met WWC evidence standards 
Wichita study 

Dynarski, M., & Wood, R. (1997). Helping high-risk youth: Results 
from the Alternative Schools Demonstration Program. Prince- 
ton, NJ: Mathematica Policy Research. 

Additional sources: 

Weinbaum, A. T., & Baker, A. M. (1991). Final implementation 
report: High School Redirection replication project. New 
York: Academy for Educational Development. 



Rubenstein, M. (1995). Giving students a second chance: 

The evolution of the Alternative Schools Demonstration 
Program. Washington, DC: Policy Studies Associates. 

Cincinnati study 

Dynarski, M., & Wood, R. (1997). Helping high-risk youth: Results 
from the Alternative Schools Demonstration Program. Prince- 
ton, NJ: Mathematica Policy Research. 



8. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within class- 
rooms or schools and for multiple comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See Technical 
Details of WWC-Conducted Computations for the formulas the WWC used to calculate the statistical significance. In the case of High School Redirec- 
tion, no corrections for clustering were needed. 
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Additional sources: 

Weinbaum, A. T., & Baker, A. M. (1991). Final implementation 
report: High School Redirection replication project. New 
York: Academy for Educational Development. 

Rubenstein, M. (1995). Giving students a second chance: 

The evolution of the Alternative Schools Demonstration 
Program. Washington, DC: Poiicy Studies Associates. 

Met WWC evidence standards with reservations 

Stockton study 

Dynarski, M., & Wood, R. (1997). Helping high-risk youth: Results 
from the Alternative Schools Demonstration Program. Princ- 
eton, NJ: Mathematica Policy Research. 



Additional sources: 

Weinbaum, A. T., & Baker, A. M. (1991). Final implementation 
report: High School Redirection replication project. New 
York: Academy for Educationai Deveiopment. 

Rubenstein, M. (1995). Giving students a second chance: 
The evolution of the Alternative Schools Demonstration 
Program. Washington, DC: Policy Studies Associates. 

Did not meet WWC evidence screens 

Baker, A. M. (1992). Using a theory of dropout prevention to 
determine the effectiveness of the High Schooi Redirection 
replication program. Dissertation Abstracts International 
52(08), 2761A. (UMI No. 9136351)^ 

Foley, E., & Crull, P. (1984). Educating the at-risk adolescent: 
More lessons from alternative high schools. A report. New 
York: Public Education Association.® 



For more information about specific studies and WWC caicuiations, piease see the WWC High School Redirection 
Technicai Appendices . 



9. Does not use a strong causal design: the study did not use a comparison group. 
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Appendix 



Appendix A1.1 Study characteristics: Dynarski & Wood, 1997 — Stockton study (randomized controiied triai with controi group crossover) 



Characteristic 


Description 


Study citation 


Dynarski, M., & Wood, R. (1997). Helping high-risk youth: Results from the Alternative Schools Demonstration Program. Princeton, NJ: Mathematica Policy Research, Inc. 



Participants The Stockton High School Redirection study used a randomized controlled trial research design. Students were assigned to the two research groups using a 2:1 random 



assignment ratio under which two students were assigned to the intervention group for every one student assigned to the control group . The original study sample of 924 
students included two cohorts. Cohort 1 consisted of students who applied to the Model Alternative High School (the replication of High School Redirection in Stockton) prior 
to or during the 1991-92 school year and included 253 students in the intervention group and 121 students in the control group. Cohort 2 consisted of students who applied 
prior to or during the 1992-93 school year and included 363 students in the intervention group and 187 students in the control group. 

The Stockton school district launched a major restructuring effort in the fall of 1993 that merged all its alternative high schools with the Model Alternative High School. This 
substantially changed the program. It also placed many control group students who had been attending other alternative high schools in the district into the intervention 
school. Because this district reorganization had a greater effect on Cohort 2 than on Cohort 1, occurring just one year after program entry for the later cohort, the study’s 
authors estimated the program’s effects using data for Cohort 1 only. Because of this disruption to the study, the WWC rated this study as meeting evidence standards with 
reservations. 

Results summarized here are based on school records, which are available for all 253 intervention students and all 121 control group students in Cohort 1 , as well as a follow- 
up survey administered three years after random assignment. Two hundred and two intervention-group students and 96 control-group students responded to the survey, for 
response rates of 80% and 79%, respectively. 

For the students in Cohort 1 , researchers compared the baseline characteristics of the two research groups on 1 2 demographic, socio-economic, and school-related 
measures. A statistical test of the overall difference between the research groups on the full set of 12 baseline characteristics found that the groups were not significantly 
different.'' 

Stockton participants were, on average, just under 17 years old at the time they applied to the High School Redirection program. Almost half (45%) were Hispanic; the rest 
were divided between whites, African-Americans, and other racial and ethnic groups. Participants were evenly split between males and females. Over half (55%) had dropped 
out of school before applying to the alternative high school. 

Setting This study took place at the Model Alternative High School, an alternative high school in Stockton, California. 

Intervention The Model Alternative High School was a replication of High School Redirection and was part of the U.S. Department of Labor’s Alternative Schools Demonstration Program 

(ASDP) evaluation. The Stockton school included most of the key features of the High School Redirection model specified by the ASDP evaluation: granting regular high school 
diplomas, taking students from throughout the district, offering the STAR remedial reading program to those with serious literacy problems, providing on-site child care, offer- 
ing no extra-curricular activities, and operating with considerable autonomy from the local district. 

During the evaluation period, the program enrolled approximately 600 students and held both morning and afternoon sessions in order to accommodate more students. Accu- 
mulating credits toward graduation was the primary goal of many of the Model Alternative students. Therefore, the school’s independent study option — under which students 
could complete assigned work on their own time and away from school — was quite popular (Rubenstein, 1995). Besides this independent study option, the school followed 
the district’s core curriculum without modification. 

(continued) 
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Appendix A1.1 


Study characteristics: Dynarski & Wood, 1997 — Stockton study (randomized controiied triai with 
controi group crossover) (continued) 


Characteristic 


Description 


Comparison 


Control group students could attend other district high schools that did not implement the High School Redirection model. In some cases, these students attended other 
alternative high schools for at-risk students operated by the district. Beginning with the 1992-93 school year, these other alternative education programs were located on the 
same campus as the Model Alternative High School. In 1993-94, these other programs were merged with Model Alternative. According to district records, in the first two 
years after random assignment (representing the 1991-92 and 1992-93 school years), 24% of control group students attended one of the alternative programs that merged 
with Model Alternative in the fall of 1993. 


Primary outcomes 
and measurement 


Five relevant outcomes from the Stockton High School Redirection study were used for rating purposes: number of days enrolled in a district high school during the first follow- 
up year, number of days enrolled in a district high school during the second follow-up year, dropped out at the end of the third follow-up year, cumulative credits earned by the 
end of the fourth follow-up year, and graduated or earned a GED certificate by the end of the third follow-up year. (See Appendices A2.1 , A2.2, and A2.3 for a more detailed 
description of outcome measures.) 


Teacher training 


Model Alternative High School teachers were regular high school teachers employed by the Stockton Public Schools. No additional information about specific training they 
received was available. 



1 . These Cohort 1 specific baseiine characteristics were not inciuded in the originai report, but were provided to the WWC by the study’s authors. 
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Appendix A1.2 Study characteristics: Dynarski & Wood, 1997 — Wichita study (randomized controiied triai) 



Characteristic 


Description 


Study citation 


Dynarski, M., & Wood, R. (1997). Helping high-risk youth: Resuits from the Aiternative Schoois Demonstration Program. Princeton, NJ: Mathematica Poiicy Research. 


Participants 


The Wichita High School Redirection study used a randomized controlled trial research design. Students were assigned to the two research groups using a 2:1 random assign- 
ment ratio under which two students were assigned to the intervention group for every one student assigned to the control group. The study sample of 358 students included 
two cohorts. Cohort 1 consisted of students who appiied to Metro-Midtown Alternative High School (the replication of High School Redirection in Wichita) prior to or during the 
1991-92 school year and included 162 students in the intervention group and 78 students in the control group. Cohort 2 consisted of students who applied prior to or during 
the 1992-93 school year and included 79 students in the intervention group and 39 students in the control group. 

Results summarized here are based on school records, which are available for all 241 intervention students and all 117 control group students in the two cohorts, as well as 
a follow-up survey administered two years after random assignment. Two hundred and two intervention-group students and 108 control-group students responded to the 
survey, for response rates of 84% and 88%, respectively. 

Researchers compared the baseline characteristics of the two research groups on 12 demographic, socio-economic, and school-related measures. A statistical test of the 
overall difference between the research groups on the full set of 12 baseline characteristics found that the groups were not significantly different. 

Wichita participants were, on average, 17 years old at the time they applied to the High School Redirection program. Almost half (45%) were white. About 4 in 10 were 
African-American and about 1 in 10 was Hispanic. Participants were evenly split between males and females. About half (53%) had dropped out of school before applying to 
the alternative high school. 


Setting 


The study was conducted at the Metro-Midtown Alternative High School, an alternative high school in Wichita, Kansas. 


Intervention 


The Metro-Midtown Alternative High School was a replication of High Schooi Redirection and was part of the U.S. Department of Labor’s Alternative Schools Demonstration 
Program (ASDP) evaluation. The Wichita school included most of the key features of the High School Redirection model specified by the ASDP evaluation: granting regular high 
school diplomas, taking students from throughout the district, offering the STAR remedial reading program to those with serious literacy problems, providing on-site child care, 
offering no extra-curricular activities, and operating with considerable autonomy from the local district. 

During the evaluation period, the program enrolled approximately 300 students and held both daytime and evening sessions. The school offered the standard district cur- 
riculum. Teachers set up individual programs of instruction for each student, and students were able to progress at their own pace (Rubenstein, 1995). 


Comparison 


Control group students could attend other district high schools that did not implement the High School Redirection model. In some cases, these students may have attended 
one of the two other alternative high schools operated by the district. Later in the evaluation, the district began admitting some control group students to Metro-Midtown 
Alternative High School if they applied again after having been originally turned away. During the first two years after random assignment, 13% of control group students spent 
some time enrolled in Metro-Midtown. 


Primary outcomes 
and measurement 


Five relevant outcomes from the Wichita High School Redirection study were used for rating purposes: number of days enrolled in a district high school during the first follow- 
up year, number of days enrolled in a district high school during the second follow-up year, dropped out by the end of the second follow-up year, cumulative credits earned 
by the end of the fourth follow-up year, and graduated or earned a GED certificate by the end of the second follow-up year. (See Appendices A2.1, A2.2, and A2.3for a more 
detailed description of outcome measures.) 


Teacher training 


Metro-Midtown Alternative High School teachers were regular high school teachers employed by the Wichita Public Schools. No additional information about specific training 
they received was available. 
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Appendix A1.3 Study characteristics: Dynarski & Wood, 1997 — Cincinnati study (randomized controiied triai) 



Characteristic 


Description 


Study citation 


Dynarski, M., & Wood, R. (1997). Helping high-risk youth: Resuits from the Aiternative Schoois Demonstration Program. Princeton, NJ: Mathematica Poiicy Research. 


Participants 


The Cincinnati High School Redirection study used a randomized controiied triai research design. Students were assigned to the two research groups using a 2:1 random 
assignment ratio under which two students were assigned to the intervention group for every one student assigned to the controi group. The study sampie of 902 students 
inciuded two cohorts. Cohort 1 consisted of students who appiied to Ciark Academy (the repiication ot High School Redirection \n Cincinnati) prior to ordering the 1993-94 
schooi year and inciuded 390 students in the intervention group and 1 85 students in the controi group. Cohort 2 consisted ot students who appiied prior to or during the 
1994-95 schooi year and inciuded 222 students in the intervention group and 105 students in the controi group. 

Resuits summarized here are based on schooi records, which are avaiiabie for aii 612 intervention students and aii 290 controi group students in the two cohorts. Research- 
ers compared the baseiine characteristics ot the two research groups on 12 demographic, socio-economic, and schooi-reiated measures. A statisticai test of the overaii 
difference between the research groups on the fuii set of 12 baseiine characteristics found that the groups were not signiticantiy different. 

Cincinnati study participants were, on average, 17.6 years oid at the time they appiied to the High School Redirection program. Most (82%) were African-American. Just over 
haif of participants (54%) were temaie. Over a third were teenage parents. About 6 in 10 had dropped out of schooi before appiying to the aiternative high schooi. 


Setting 


The study was conducted at the Peter H. Ciark Academy, an aiternative high schooi in Cincinnati, Ohio. 


Intervention 


The Peter H. Ciark Academy was a repiication of High School Redirection and was part of the U.S. Department of Labor's Aiternative Schoois Demonstration Program (ASDP) 
evaiuation. The Cincinnati schooi inciuded most of the key features of the High Schooi Redirection modei specified by the ASDP evaiuation: granting reguiar high schooi dipio- 
mas, taking students from throughout the district, offering the STAR remediai reading program to those with serious iiteracy probiems, offering no extra-curricuiar activities, 
and operating with considerabie autonomy from the iocai district. 

The program had capacity for 250 students and operated out of a converted eiementary schooi in a poor neighborhood of Cincinnati. Uniike the other High Schooi Redirection 
repiication sites summarized in this report, the program did not offer on-site chiid care, instead, it offered off-site chiid care and van service between the schooi and the 
daycare faciiity. The schooi deveioped a speciai curricuium for ninth grade in which educationai themes were integrated across the curricuium. The schooi aiso offered a modi- 
fied Engiish curricuium that aiiowed some students to earn graduation credits at an acceierated pace. Beyond these two modifications, the schooi offered the standard district 
curricuium (Rubenstein, 1995). 


Comparison 


Controi group students couid attend other district high schoois that did not impiement the High School Redirection modei. At the time of the ASDP evaiuation, the Cincinnati 
Pubiic Schoois offered few other aiternative education programs for at-risk students besides the Peter H. Ciark Academy. 


Primary outcomes 
and measurement 


Two reievant outcomes from the Cincinnati High School Redirection study were used for rating purposes: dropped out at the end of the second foiiow-up year and graduated or 
earned a GED certificate by the end of the second foiiow-up year. (See Appendices A2.1, A2.2, and A2.3 for a more detaiied description of outcome measures.) 


Teacher training 


Ciark Academy teachers were reguiar high schooi teachers empioyed by the Cincinnati Pubiic Schoois. No additionai information about specific training they received was 
avaiiabie. 
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Appendix A2.1 Outcome measures for the staying in schooi domain 



Outcome measure 


Description 


Number of days 
enrolled in year 1 


This measure represents the number ot days that students were enrolled in a district high school during the first follow-up year. These data were collected through school 
records for the Stockton and Wichita studies. This measure was not available in the Cincinnati study. 


Number of days 
enrolled in year 2 


This measure represents the number of days that students were enrolled in a district high school during the second follow-up year. These data were collected through school 
records for the Stockton and Wichita studies. This measure was not available in the Cincinnati study. 


Dropped out 


This binary measure represents whether a student has dropped out of school. In the Stockton and Wichita studies, this outcome is based on follow-up survey data and is 
defined as not being enrolled in any school or GED program and not having earned a high school diploma or GED certificate. For the Stockton study, the outcome is measured 
at the end of year 3; for the Wichita study, it is measured at the end of year 2. In the Cincinnati study, this outcome is based on school records data, as well as GED testing 
data maintained by the Ohio Department of Education. It is measured at the end of year 2 and represents not being enrolled in a district high school and not having earned a 
district diploma or a GED certificate in the state. It does not include enrollment in a GED program. 



Appendix A2.2 Outcome measures for the progressing in schooi domain 



Outcome measure 


Description 


Total credits earned 


This measure represents the cumulative number of credits that students earned by the end of the fourth follow-up year. For the Stockton and Wichita studies, these data were 
collected through school records. This measure was not available for the Cincinnati study. 



Appendix A2.3 Outcome measures for the compieting schooi domain 



Outcome measure 


Description 


Earned a high school 
diploma or GED 


This binary measure represents the percentage of students that earned a high school diploma or GED certificate at a particular point after random assignment. In the Stockton 
and Wichita studies, this outcome is based on follow-up survey data and is defined as having earned a diploma from any high school or a GED certificate in any state. For the 
Stockton study, the outcome is measured at the end of year 3; for the Wichita study, it is measured at the end of year 2. In the Cincinnati study, this outcome is measured at 
the end of year 2 and represents having graduated from a district high school or having earned a district diploma or a GED certificate in Ohio. The measure is based on school 
records data, as well as GED testing data that were provided by the Ohio Department of Education. 
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Appendix A3.1 



Summary of study findings inciuded in the rating for the staying in schooi domain^ 









Author’s findings from the study 














Mean outcome 
(standard deviation^) 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


High School 

Redirection Comparison 

group group 


Mean difference^ 
{High School 
Redirection - 
comparison) 


Statistical 
significance^ 
Effect size^ (at a = 0.05) 


Improvement 

index^ 



Number of days 
enrolled In year 1 


Dynarski and Wood, 1997 (randomized controlled trial with control group crossover)^ — Stockton 

Cohort 1 374 110 71 39 0.53 

(71) (78) 


Statistically 

significant 


+20 


Number of days 
enrolled In year 2 


Cohort 1 


374 


67 

(78) 


50 

(73) 


17 


0.22 


Statistically 

significant 


+9 


Dropped out at end of year 3 (%) 


Cohort 1 


298 


43 


53 


10 


0.24 


ns 


+9 


Average^ for staying in school (Dynarski and Wood, 1997) — Stockton 








0.33 


Statistically 

significant 


+13 






Dynarski and Wood, 1997 (randomized controlled trial)^ — Wichita 








Number of days 


Cohorts 1 and 2 


358 


108 


94 


14 


0.21 


ns 


+8 


enrolled In year 1 






(64) 


(73) 










Number of days 


Cohorts 1 and 2 


358 


59 


60 


-1 


-0.01 


ns 


0 


enrolled In year 2 






(77) 


(73) 










Dropped out at end of year 2 (%) 


Cohorts 1 and 2 


310 


51 


46 


-5 


-0.12 


ns 


-5 


Average^ for staying in school (Dynarski and Wood, 1997) — Wichita 








0.03 


ns 


+1 






Dynarski and Wood, 1997 (randomized controlled trial)^ — Cincinnati 








Dropped out at end of year 2 (%) 


Cohorts 1 and 2 


902 


80 


83 


3 


0.12 


ns 


+5 


Average^ for staying in school (Dynarski and Wood, 1997) — Cincinnati 






0.12 


ns 


+5 


Domain average^ for staying in school 










0.16 


na 


+6 



ns = not statistically significant 
na = not applicable 

1 . This appendix reports findings considered for the effectiveness rating and the average improvement index in the staying in school domain. 

2. Standard deviations are reported for continuous outcomes only. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would 
indicate that participants had more similar outcomes. Standard deviations for number of days enrolled were not included in Dynarski and Wood (1997), but were reported to the WWC by the study's authors. 

3. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. For the dropout outcome, signs were reversed on the mean difference, effect size, and improvement 
index to demonstrate that the treatment group was favored when negative differences were reported. Means from Dynarski and Wood (1997) are estimated using regression models that control for baseline characteristics. 

4. The effect sizes for dichotomous variables were computed using the Cox Index. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

(continued) 
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Appendix A3.1 Summary of study findings inciuded in the rating for the staying in schooi domain (continued) 



5. Statistical significance is the probability that the difference between the groups is a result of chance rather than a real difference between the groups. Although the study authors originally reported statistical significance using a 
p < 0.10 threshold, all analyses were redone by the WWC to test for statistical significance using a p < 0.05 threshold. 

6. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

7. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clus- 
tering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of the Dynarski and Wood (1997) Stockton, 
Wichita, and Cincinnati studies, no corrections for clustering were needed. 

8. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect sizes. 
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Appendix A3.2 Summary of study findings inciuded in the rating for the progressing in schooi domain^ 









Author’s findings from the study 

Mean outcome 
(standard deviation^) 




WWC calculations 














Mean difference^ 














High School 




{High School 




Statistical 






Study 


Sample size 


Redirection 


Comparison 


Redirection - 




significance^ 


Improvement 


Outcome measure 


sample 


(students) 


group 


group 


comparison) 


Effect size^ 


(at a = 0.05) 


index^ 



Dynarski and Wood, 1997 (randomized controlled trial with control group crossover)^— 


-Stockton 






Total credits earned Cohort 1 374 10.5 8.5 2.0 

at end of year 4 (7.6) (7.4) 


0.26 


Statistically 

significant 


+10 


Average^ for progressing in school (Dynarski and Wood, 1997) — Stockton 


0,26 


Statistically 

significant 


+10 


Dynarski and Wood, 1997 (randomized controlled trial)^ — Wichita 








Total credits earned Cohorts 1 and 2 358 7.6 8.2 -0.6 

at end of year 4 (6.9) (6.8) 


-0,09 


ns 


-3 


Average^ for progressing in school (Dynarski and Wood, 1997) — Wichita 


-0,09 


ns 


-3 


Domain average^ for progressing in school 


0,09 


na 


+4 



ns = not statistically significant 

na = not applicable 

1 . This appendix reports findings considered for the effectiveness rating and the average improvement index in the progressing in school domain. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. Standard 
deviations for total credits earned were not included in Dynarski and Wood (1997), but were reported to the WWC by the study’s authors. 

3. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. Means from Dynarski and Wood (1997) are estimated using regression models that control for baseline 
characteristics. 

4. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

5. Statistical significance is the probability that the difference between the groups is a result of chance rather than a real difference between the groups. Although the study authors originally reported statistical significance using a 
p < 0.10 threshold, all analyses were redone by the WWC to test for statistical significance using a p < 0.05 threshold. 

6. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

7. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clus- 
tering correction, see the WWC Tutorial on Mismatch . See echnical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of the Dynarski and Wood (1997) Stockton 
and Wichita studies, no corrections for clustering or multiple comparisons were needed. 

8. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect sizes. 
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Appendix A3.3 Summary of study findings inciuded in the rating for the compieting schooi domain^ 









Author’s findings from the study 
Mean outcome 




WWC calculations 














Mean difference^ 














High School 




(High School 




Statistical 






Study 


Sample size 


Redirection 


Comparison 


Redirection - 




significance'* 


Improvement 


Outcome measure 


sample 


(students) 


group 


group 


comparison) 


Effect size^ 


(at a = 0.05) 


index^ 



Dynarski and Wood, 1997 (randomized controlled trial with control group crossover)^- 


—Stockton 






Earned a high school diploma Cohort 1 298 40 32 8 

or GED at end of year 3 (%) 


0.21 


ns 


+8 


Average^ for completing school (Dynarski and Wood, 1997) — Stockton 


0.21 


ns 


+8 


Dynarski and Wood, 1997 (randomized controlled trial)^ — Wichita 








Earned a high school diploma Cohorts land 2 310 29 30 -1 

or GED at end of year 2 (%) 


-0.03 


ns 


-1 


Average^ for completing school (Dynarski and Wood, 1997) — Wichita 


-0.03 


ns 


-1 


Dynarski and Wood, 1997 (randomized controlled trial)^ — Cincinnati 








Earned a high school diploma Cohorts 1 and 2 902 7 6 1 

or GED at end of year 2 (%) 


0.10 


ns 


+4 


Average^ for completing school (Dynarski and Wood, 1997) — Cincinnati 


0.10 


ns 


+4 


Domain average^ for completing school 


0.09 


na 


+4 



ns = not statistically significant 

na = not applicable 

1 . This appendix reports findings considered for the effectiveness rating and the average improvement index for the completing school domain. Appendix A4 reports separate impacts of High School Redirection on earning a high school 
diploma and earning a GED certificate. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. Means from Dynarski and Wood (1997) are estimated using regression models that control for baseline 
characteristics. 

3. Effect sizes for dichotomous variables were computed using the Cox Index. For an explanation of the effect size calculation, see Technical Details of WWC-Conducfed Computations . 

4. Statistical significance is the probability that the difference between the groups is a result of chance rather than a real difference between the groups. Although the study authors originally reported statistical significance using a 
p < 0.10 threshold, all analyses were redone by the WWC to test for statistical significance using a p < 0.05 threshold. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clus- 
tering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of the Dynarski and Wood (1997) Stockton, 
Wichita, and Cincinnati studies, no corrections for clustering or multiple comparisons were needed. 

7. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect sizes. 
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Appendix A4 Summary of item-level findings for the completing school domain^ 









Author’s findings from the study 
Mean outcome 




WWC calculations 














Mean difference^ 














High School 




(High School 




Statistical 






Study 


Sample size 


Redirection 


Comparison 


Redirection - 




significance'* 


Improvement 


Outcome measure 


sample 


(students) 


group 


group 


comparison) 


Effect size^ 


(at a = 0.05) 


index^ 





Dynarski and Wood, 1997 (randomized controlled trial with control group crossover)- 


—Stockton® 






Earned a high school diploma 
at end of year 3 (%) 


Cohort 1 


298 


20 


15 


5 


0.21 


ns 


-h8 


Earned a GED certificate 
at end of year 3 (%) 


Cohort 1 


298 


21 


17 


4 


0.16 


ns 


-h6 






Dynarski and Wood, 1997 (randomized controlled trial)- 


—Wichita® 








Earned a high school diploma 
at end of year 2 (%) 


Cohorts 1 and 2 


310 


9 


14 


-5 


-0.30 


ns 


-12 


Earned a GED certificate 
at end of year 2 (%) 


Cohorts 1 and 2 


310 


20 


17 


3 


0.12 


ns 


-h5 






Dynarski and Wood, 1997 (randomized controlled trial)— 


-Cincinnati® 








Earned a high school diploma 
at end of year 2 (%) 


Cohorts 1 and 2 


902 


4 


1 


3 


0.86 


Statistically 

significant 


-h30 


Earned a GED certificate 
at end of year 2 (%) 


Cohorts 1 and 2 


902 


3 


5 


-2 


-0.32 


ns 


-13 



ns = not statistically significant 

1. This appendix presents item-level findings for measures in fhe completing school domain. These include the separate effects of the intervention on high school diploma receipt and on GED certificate receipt. The intervention’s com- 
bined effect on high school diploma and GED receipt was used for determining the effectiveness rating and is presented in Appendix A3. 3. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. Means from Dynarski and Wood (1997) are estimated using regression models that control for baseline 
characteristics. 

3. Effect sizes for dichotomous variables were computed using the Gox Index. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

4. Statistical significance is the probability that the difference between the groups is a result of chance rather than a real difference between the groups. Although the study authors originally reported statistical significance using a 
p < 0.10 threshold, all analyses were redone by the WWC to test for statistical significance using a p < 0.05 threshold. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and the average student in the comparison condition. The improvement index can take on values 
between -50 and -i-50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clus- 
tering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations tor the formulas the WWC used to calculate statistical significance. In the case of the Dynarski and Wood (1997) Stockton, 
Wichita, and Cincinnati studies, no corrections for clustering or multiple comparisons were needed. 
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Appendix A5.1 High School Redirection rating for the staying in schooi domain 



The WWC rates an intervention’s effects in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.^ 

For the outcome domain of staying in school, the WWC rated High School Redirection as having mixed effects. It did not meet the criteria for positive effects 
because oniy one of the three studies showed positive effects. It did not meet the criteria for potentiaiiy positive effects because more studies found indeterminate 
effects than found significant positive effects. The remaining ratings (no discernable effects, potentially negative effects, negative effects) were not considered because 
High School Redirection was assigned a higher applicable rating. 



Rating received 

Mixed effects: Evidence of inconsistent effects as demonstrated through EITHER of the following criteria. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No study found statistically significant or substantively important negative effects in this domain. 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Met. One study found a statistically significant positive effect in this domain and two studies had indeterminate effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Only one study found statistically significant positive effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No study found statistically significant or substantively important negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Met. One study found statistically significant and substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Not met. Two studies showed indeterminate effects while only one showed statistically significant positive effects. 

1 . For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A5.2 High School Redirection rating for the progressing in schooi domain 



The WWC rates an intervention’s effects in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.^ 

For the outcome domain of progressing in schooi, the WWC rated High School Redirection as having potentially positive effects. It did not meet the criteria for hav- 
ing positive effects because oniy one study found statistically significant or substantiveiy important positive effects in this domain. The remaining ratings (mixed effects, 
no discernabie effects, potentiaiiy negative effects, negative effects) were not considered because High School Redirection was assigned a higher applicable rating. 

Rating received 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Met. One study found statistically significant and substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Met. No study found statistically significant or substantively important negative effects in this domain. One study showed statistically significant 
positive effects and another showed indeterminate effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. One study found statistically significant positive effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No study found statistically significant or substantively important negative effects in this domain. 

1 . For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A5.3 High School Redirection rating for the compieting schooi domain 



The WWC rates an intervention’s effects in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.^ 

For the outcome domain of completing schooi, the WWC rated High School Redirection as having no discernibie effects. It did not meet the criteria for the other 
ratings (positive effects, potentially positive effects, mixed effects, potentially negative effects, negative effects) because no statistically significant or substantively 
important findings were found in this domain. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Met. No study found statistically significant or substantively important effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No study found statistically significant positive effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No study found statistically significant or substantively important negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Not met. No study found statistically significant or substantively important positive effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Met. No study found statistically significant or substantively important negative effects in this domain. 

Mixed effects: Evidence of inconsistent effects as demonstrated through EITHER of the following criteria. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No study found statistically significant or substantively important effects, either positive or negative, in this domain. 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No study found statistically significant or substantively important effects in this domain. 

(continued) 
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Appendix A5.3 High School Redirection rating for the compieting schooi domain (continued) 



Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence 

• Criterion 1: At least one study showing a statisticaiiy significant or substantively important negative effect. 

Not met. No study found statistically significant or substantively important negative effects in this domain. 

• Criterion 2: No studies showing a statistically significant or substantively important positive effect, or more studies showing statistically significant or substantively 
important negative effects than showing statistically significant or substantively important positive effects. 

Met. No study found statistically significant or substantively important positive effects in this domain. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No study found statistically significant negative effects in this domain. 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No study found statistically significant or substantively important positive effects in this domain. 



1 . For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A6 Extent of evidence by domain 



Outcome domain 


Number of studies 


Schools 


Sample size 

Students 


Extent of evidence^ 


Staying in school 


3 


3 


1,634 


Moderate to large 


Progressing in school 


2 


2 


732 


Moderate to large 


Completing school 


3 


3 


1,510 


Moderate to large 



na = not applicable/not studied 

1 . A rating of “moderate to large” requires at least two studies and two schools across studies In one domain, and a total sample size across studies of at least 350 students or 14 classrooms. 
Otherwise, the rating is “small.” 
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